A Model-Based Experiment towards an Emotional Syn- thesis

The most successful methods to induce emotions on state of the art unit selection speech synthesis have been built by switching speech database depending on the desired emotion. These methods require a substantial increase of memory compared to a single database and are computationally slow. The model-based approach is an attempt to reshape a neutrally recorded utterance (comparable to the desired output from a modern unit selection system) into simulating a recorded model of a desired emotion. Factors for manipulation of duration, amplitude and formant shift ratio are calculated by comparing the recorded neutral utterance with three recorded, basic emotional models in accordance with discrete emotion theory – sadness, happiness and anger. F0 (regarded as the intonation) is copied from the model and is then imposed on the neutrally recorded utterance. The evaluation of the experiment shows that subjects easily categorize discrete emotions in a forced choice. They also grade the resynthesized emotional quality from the neutrally recorded utterance almost equally high as the naturally recorded models for the male voice. The female voice created more difficulties and contained more synthetic artifacts, i.e. it was judged to have a lower quality than the recorded models. Background and Introduction Creating emotional synthesis has been a research area for quite some time. Formant speech synthesis is easily distinguished from human speech not only because of the underdeveloped naturalness, but also due to the lack of expressiveness. Several attempts to implement emotions in formant synthesis have taken place (Cahn, 1988; 1989; 1990; Carlson et al., 1992). When dealing with emotional content in speech the point of departure is almost always the neutral utterance. What is neutral speech, i.e. speech without emotions? Normally, neutral speech is thought of as a carrier being modulated to reveal the emotions being communicated. Such a concept is rather useful when it comes to synthesizing expressive speech. One simply treats the relationship in a hierarchy where the abstract underlying expression is neutral and the surface expressions are the emotions we want to induce, in this case the basic emotions from discrete emotional theory anger, sadness and happiness (Levenson, 1994; Laukka, 2004; Tatham & Morton, 2004; Narayanan & Alwan, 2004). A modern state of the art unit selection speech synthesis normally produces a sentence as neutrally as possiblein order to avoid undesired side effects or miscommunication. Neutral in this case means near monotone or containing as few speech fluctuations as possible. This is not always desirable when it comes to for example dialogue systems. To be able to compare whether a system succeeds in expressing a certain emotion or desire, it is obviously also important to study how well people in general succeed in communicating emotions. The development of conversational systems has increased, meaning that understandable, neutral synthetic speech is barely acceptable anymore. Some success has been reached, but the best ones still depend too much on stored data, including a separate emotional speech database. (Bulut et al., 2002) The most successful attempts to synthesize emotions have been built by using additional speech databases containing only recordings representing specific emotions uttered (this applies to concatenative/unit selection synthesis systems). The system has to be able to switch database when a specific emotion is desirable. The system must perhaps also use different algorithms/analyses for the different databases since the acoustic content might differ significantly. The databases needed for such a system also mean a substantial increase of data to choose from. A simpler and computationally more efficient method is to induce rules for expressive speech and resynthesize an utterance produced by the system. Nowadays, most unit selection systems are created by recording a single professional speaker and then using specified parts (nor-